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ABSTRACT 

This study is aimed at investigating students’ annotation behaviors and their effects on learning achievement with pre- 
reading (reading before class) exercises and Web-based sharing mechanisms. An 8-week quasi-experiment was 
conducted with 125 sixth-grade elementary school students. The study shows the following results. First, a significant 
correlation was evident between pre-reading with a Web-based annotation tool and learning achievement. Furthermore, 
implementing a pre-reading sharing mechanism with Web-based annotation capabilities could stimulate and help 
students perform more useful pre-reading by reviewing others’ annotations, thereby enlarging the effectiveness of pre- 
reading as it relates to learning. Second, participants’ pre-reading exercises with a Web-based annotation tool revealed 
their preparation before class as well as reflected their prior knowledge, thereby helping the instructor to prepare the 
lecture well in advance. In the future, more advanced mechanisms, such as self-regulation learning, will be studied to 
encourage students to efficiently manage learning before class. 

Keywords: Pre-reading; Annotations; Improving classroom teaching 

INTRODUCTION 

Students engage in an especially helpful and useful practice by making annotations, such as comments or explanations, 
to learning materials. In particular, the use of a Web-based annotation tool during lectures or after class has positive 
influences on students’ learning achievement and it stimulates students’ motivation to learn (Hwang, Wang & Sharpies, 
2005). Pre-reading appears to be a good way to help students prepare for learning in class because students’ readiness to 
learn is an important factor to achieve effective learning (Zhang, 2001). However, few studies have concentrated on the 
pre-reading with a Web-based annotation mechanism and how this mechanism affects learning achievements. In this 
study, we investigate whether learning before class (pre-reading) with a Web-based annotation mechanism could help 
students perform significantly better than students who did not engage in pre-reading exercises or pre-read with 
traditional textbook. Learner annotations written in class could be a useful way to assess students' understanding about 
the lecture (Lin, 2006). Thus, in this study, we determine whether learner annotations written before class could help 
teachers understand students’ prior knowledge and preparation before class. Furthermore, we study whether a sharing 
mechanism could facilitate learner pre-reading skills and facilitate more effective annotations in a Web-based 
environment. 

LITERATURE 

Influence of pre-reading on learning achievement 

Pre-reading refers to reading or studying before class. In other words, pre-reading conveys one’s readiness before class. 
“The most important single factor influencing learning is what the learner already knows” (Ausubel, 1978, p. 163). 
Additionally, “ a primary process in learning is subsumption in which new material is related to relevant ideas in the 
existing cognitive structures" (Kearsley, 2000). Thus, meaningful pre-reading refers to being prepared before learning 
takes place, while trying to relate new learning material to previous knowledge (Zhang, 2001). The process of making 
annotations helped students to connect lecture content with prior knowledge (Peper & Mayer, 1978), including 
integrating new concepts of a lecture into meaningful learning (Ausubel, 1968) and generative learning (Wittrock, 
1974). Paul (1979) also indicated that making preparations is an effective way to help create “learning-ready” students 
and Chan (2005) indicated that the preparations for the pre-test, which were held 10 minutes before each class, could 
stimulate students to work hard and gain some knowledge of the learning materials before class. Qiao and Zhao (2008) 
developed a Web-based pre-test to improve teaching quality. However, the above studies were mainly concerned that 
the learning effects resulted from the students' prior knowledge in the field, and were not related to the pre-reading 
strategy. In addition, the two studies did not further uncover students’ pre-reading behaviors such as the quantities and 
types of note-taking. 

Spies and Wilkin (2004) showed that before each class, the students who were responsible for reading a legal case 
displayed a greater understanding of the learning materials than students who were not expected to prepare for a 
pharmacy law course. Chiu and Lee (2009) considered that a pre-class video viewing of the lecture content and hands- 
on laboratory activities in class enhanced the learning of high-school students’ basic image processing. Chen (2008) 
showed that pre-reading helped learners capture key points and incomprehensible concepts before class, so they could 
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focus their attention on the parts of the lecture related to those key points and concepts previously not fully understood. 

The findings in the Chung and Fan study (2007) showed that when a teacher requested that students pre-read before 
class and did not plan activities well to stimulate pre-reading, it was found that some students did not attain good 
learning readiness for the lecture and did not perform well on exams. Sun and Huang (2005) conducted an experiment 
with an experimental group and a control group, employing traditional instruction with or without asking for pre- 
reading Web-based learning materials. The results indicated that the pre-reading group received significantly higher 
scores than the group that did not pre-read. Unfortunately, the study did not further investigate the factors leading to the 
results, such as the strengths of Internet-like sharing and any detailed engagement students were involved in before 
class. As previously mentioned, learning readiness is an important factor and worth studying in detail with well- 
designed activities or mechanisms to stimulate additional pre-reading. 

Influence of annotation on learning performance 

Annotation refers to marking extra information on reading documents. According to Marshall (1997), annotation can be 
divided into two parts: explicit annotation and inexplicit annotation. The former (e.g., text) conveys more explicit 
meaning than the latter (e.g., highlighting, underlining, asterisks, arrows, and graphics) for the original annotator, which 
conveys the common meaning (Marshall, 1998). According to these different forms of annotations, Marshall (1997) 
proposed six annotative functions for the original annotator to use during later reviewing. Inexplicit annotation features 
procedural function signals for marking an area already known, or for a desire to know, place-marking for aiding 
memory, and drawing for a situ method of a working problem. Explicit annotation, however, functions as short notes 
that may combine the inexplicit for interpretation, reflections, or for a visible trace of a reader’s attention (Marshall, 
1997). 

Literature, such as the following, showed that annotation has the potential to facilitate the effectiveness of learning. 
Annotations positively affected learning achievement; the more annotations were made, the greater learning 
achievements were obtained (Petri, Miikka, Jaako, Patrik & Henry, 2005). Meanwhile, annotating learning material is 
an effective learning strategy to promote students’ reading comprehension (Chang, Chen & Chen, 2006) by highlighting 
or underlining key concepts (Shaughnessy & Bake, 1988) and it effectively reduces the readers’ cognitive overload by 
allowing them to write short notes related to reading materials (Marshall, 1998). Finding key concepts and the 
supporting related facts in the material have been regarded as the necessary basic skills for reading comprehension and 
summarization capabilities (Zimmerman, 1986). 

With the growth in digital learning activities and the Internet, annotation on Web-based learning materials has gradually 
attracted worldwide attention. Yeh and Lo (2009) presented an experiment and developed a Web-based interactive 
system, called Online Annotator for EFL Writing, for giving feedback on second language writing of college freshmen 
by marking annotations. The results of the study showed that students who were arranged in a Web-based annotation 
tool group have significantly better error recognition learning performance than those who were arranged in a paper- 
based annotation group. This enhanced performance was due to the limited amount of corrective and feedback 
information displayed in annotations shared by a teacher or peers (Yeh & Lo, 2009). 

Su, Yang, Hwan and Zhang (2010) conducted a study and designed a Web 2.0 collaborative annotation system, called 
the Personalized Annotation Management System 2.0, to examine the relationship between learning achievements and 
the quantity of annotations created by college freshmen during or after a lecture. The study revealed that the annotation 
sharing mechanism is a key to strengthen students’ learning achievements (Su, Yang, Hwan & Zhang, 2010). 

According to the previous studies, annotations created by students during lectures or after class have positively 
correlated with students' learning achievement. However, few studies further explore the effectiveness of annotations 
created before class on learning achievement, and the correlation between learning achievement and annotations created 
before class. 

Perceived usefulness and system ease of use and activity design with an annotation sharing mechanism 

Perceived usefulness and perceived ease of use proposed by Davis (1986) have been widely used to predict user 
attitudes toward information technology (Chang & Yang, 2010; Park, 2010; Selim, 2003). Perceived usefulness refers to 
the belief that using an information system will increase and improve their performance. Perceived ease of use refers to 
the belief that using an information system will be free of effort (Selim, 2003). A person’s behavior toward an 
information system was determined by his attitude concerning perceived usefulness and perceived ease of use (Davis, 
1986). 

Recently, online interpersonal knowledge sharing networks have become popular, like Facebook, Twitter, etc. As for 
reviewing annotations marked by other students, Hicks (2003) emphasized that shared annotations have an advantage of 
allowing for informal sharing of personal knowledge related to an artifact or concept. Sharing mechanisms have been 
widely applied in Web-based learning environments to promote the following effects: to stimulate the motivation to 


Copyright © The Turkish Online Journal of Educational Technology 


235 



TOJET: The Turkish Online Journal of Educational Technology - April 2011, volume 10 Issue 2 


involve students in learning activities and help them move forward toward learning goals (Hwang, Wang & Sharpies, 
2005; Silvia & Andy, 2003); to achieve better learning performance by collaborating with peers by sharing and 
accessing their own ideas of learning materials (Hwang, Wang & Mike, 2007; Su, Yang, Hwang & Zhang, 2010); to 
obtain the benefits of peer learning, such as offering more learning opportunities through conversation or dialogue via 
sharing annotations (Glover, Xu & Hardaker, 2007; Wolfe, 2002); to learn a peer’s methods of how to accomplish a 
specific task via sharing and discussing their annotations with other peers (Cobos & Pifarre, 2008); and to support 
learner-center collaborative learning for adult and adolescent students via a shared document-based annotation tool 
(Nokelainen, Miettinen, Kurhila, Floreen & Tirri, 2005). Thus, making annotations is not only helpful for enhancing 
individual learning, but it is also a useful way to facilitate peer learning through sharing annotations. 

Therefore, in this study the perceived usefulness and perceived ease of using a Web-based annotation system, called 
Virtual PEN (VPEN), with a pre-reading stimulus were employed to explore student attitudes toward our proposed 
activities and system. Furthermore, the effect of the pre-reading stimulus using VPEN, with/without sharing for 
learning, was deeply investigated and the reasons behind it were also analyzed. 

RESEARCH METHOD 
Research objectives 

The purpose of this study is to investigate the effects of pre-reading modes with/without a sharing mechanism and 
annotation tools on learning achievement. Meanwhile, students and the instructor perceived the role of pre-reading and 
the sharing mechanism on learning. Three research objectives were proposed. 

(1) . The effects of the pre-reading modes, with the Web-based annotation tool or with traditional paper and pen, on 

learning achievement were investigated. 

(2) . Pre-reading modes with a Web-based sharing mechanism were conducted along with their influence on 

learning achievement and the quantity of annotations. 

(3) . The students’ and instructor’s perceptions about pre-reading, the VPEN tool, and the sharing scenario were 

explored. 

Research participants and subject 

Four classes, a total of 125 sixth-grade elementary school students, participated in a total of two phases of this 
experiment. Shown in Table 1, four classes are divided by the pre-reading and annotation tool into four segments. The 
left column (“Experiment” and “Control B”) asked for pre-reading while the right column did not require pre-reading 
(“Control A” and “Control C”). The top row (“Experiment” and “Control A”) employed the VPEN tool while the 
bottom row (“Control B” and “Control C”) employed a traditional paper and pen approach. The subject in this study is a 
social course in a sixth-grade elementary school and the learning material in this study is the textbook. 


Table 1 :Pre-reading Modes and Tools 


TooEPre-reading Modes 

Pre-reading Group 

Group Not Asked to Pre-read 

Web-based learning materials 
with VPEN for annotation 

Experiment: 30 students 

Control A: 32 students 

Textbook with paper and pen for 
annotation 

Control B: 31 students 

Control C: 32 students 


Research variables and structure 

The independent variables of this research are (1) pre-reading, (2) the annotation tool, and (3) the sharing scenario. The 
dependent variables in this research are (1) learning achievement and (2) the quantity of annotation. The same teacher 
taught the all participants. The participants also learned the same material (the “Experiment” and “Control A” learned 
from the Web-based version of textbook, while the “Control B" and “Control C” learned from the paper-based 
textbook) within the same learning period and schedule. Meanwhile, the statistical method of Analysis of Covariance 
(ANCOVA) was employed to exclude the difference of participants’ prior knowledge between the groups; these are the 
controlled variables in this research. The research variables include the followings. 

(1) Pre-reading: In this study, pre-reading refers to studying before class. See Table 1 (“Experiment” and “Control 
B” constitute the pre-reading group, which asked to pre-read learning materials before class while “Control A” 
and “Control C" did not ask). 

(2) Annotation tool: The annotation tool consists of VPEN annotation and paper-based annotation. The former, the 
VPEN tool, was a Web-based annotation system while the latter used a pen to annotate paper-based learning 
materials (“Experiment” and “Control A” are the VPEN annotation group. “Control B” and “Control C” are the 
paper-based annotation group). 

(3) Sharing scenario: The Web-based annotation tool with a sharing annotation mechanism was provided for the 
“Experiment” and “Control A” to support student learning and pre-reading for the social course in phase 2. 
Figure 4 is an illustration of the sharing mechanism for viewing classmates’ annotations. 

(4) The quantity of annotations: The total number of annotations created before class, in class, and after class. 
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which consisted of explicit and inexplicit annotations. 

(5) Participants’ prior knowledge: The scores of the pre-test measured prior knowledge. 

(6) Learning achievement: The scores of the post-test measured learning achievement. 

(7) 

Figure 1 illustrates the structure of the research variables. Research objective one, mentioned in Section, “Research 
objectives ”, was to identify the effects of two variables (pre-reading and annotation tool) on learning achievement by 
two-way ANOCVA (lines 2 and 3). Furthermore, Pearson’s correlation was employed to further investigate the 
correlation between achievement and the quantity of explicit or inexplicit annotations made before class. According to 
research objective two, mentioned in Section, “Research objectives”, the sharing mechanism was explored to identify 
its influence on learning achievement by two-way ANOCVA (line 5). Furthermore, research objective two was also 
instituted to explore whether there is a significant difference in the quantity of annotations between the classes 
with/without pre-reading by T-test in two phases without/with a sharing mechanism (lines 4 and 6). In the third research 
objective, the attitudes of students and the instructor toward the system and the proposed activities were explored via 
interviews or questionnaires. 


Control variable: 



Figure 1. Research variables and structure 


Experimental procedures 

An 8-week quasi-experiment, with each week devoting 2 hours to the research, was conducted. The intervention is 
divided into two phases without or with a sharing mechanism to identify the effects of pre-reading, the annotation tool, 
and the sharing annotation mechanism on learning and the attitudes of the instructor and students toward them. The 
experimental procedures are described as follows (and illustrated as Figure 2). 

(1) .Training activities: Training and practices are conducted for the use of the VPEN tool in the VPEN group and 

for pre-reading in the pre-reading group before the first phase. The training of the use of a Web-based 
annotation sharing mechanism was conducted in the VPEN group before the second phase. The learning 
materials used in the training activity are the lessons of the textbook in the social course, which are not part 
of the learning materials used in intervention. The training of the VPEN tool is focused on how to 
login/logout of VPEN, annotating the learning material in explicit or inexplicit annotations, and how to save 
the annotations. The training of the Web-based annotation sharing mechanism is focused on how to view the 
annotations made by classmates in phase 2. The activities of pre-reading include guiding questions, which 
were given by the teacher to facilitate students to annotate and to think about main ideas and the learning 
materials’ incomprehensible concepts before class. Meanwhile, the students were asked to label their 
annotations as “uncomprehending idea,” “main idea,” etc. 

(2) .Pre-test: Before each phase, a total of two pre-tests were held for four classes to measure the students' prior 

knowledge. 

(3) .Intervention (two phases): Four classes are assigned the same learning activities and the same instructor during 

a lecture. The learning activities in class include the following: guiding questions at the beginning of a 
lecture, the instructor’s lecture, and peer discussion. Only the pre-reading group asked to pre-read before 
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class. The pre-reading and training activities are the same. In the first phase, the Web-based annotation tool 
and pre-reading modes were employed without a sharing mechanism. In the second phase, for the VPEN 
group, the “Experiment” with pre-reading and “Control A” without pre-reading are conducted with the 
sharing mechanism while the paper-based annotation group, “Control B” and “Control C,” are not prohibited 
nor encouraged to share annotations made in textbooks. 

(4) .Post-test: After each phase, a total of two post-tests are held for four classes to measure the students’ learning 

achievements. The post-tests are held 2 weeks after the intervention. 

(5) .Questionnaire and interview: At the end of the experiment, a questionnaire or interview are held in order to 

identify the instructor’s and students’ perceptions toward pre-reading, the VPEN tool, and the sharing 
mechanism. 

(6) .Data collection and analysis: The collected data depend on research objectives. Thus, the data include the 

scores of the pre-test or post-test, the quantity of annotations that were calculated by the concept-based 
coding, the questionnaire data, and the interview data. The analytical method of these data has been 
proposed in Section, “Research variables and structure”. 


Training: 

Phase 1: Training the use ol VPEN lor VPEN group and Pre-Reading (or Pre-Reading group 
Phase 2: Training the use of web-based sharing mechanism (or VPEN group 


0 ) 

t 


Pre-Test: 

Phase 1: pre-test 
Phase 2: pre-test 

- 1 - 

( 2 ) 

_ 1 _ 



T 

(3) 



Figure 2. Experiment procedures 


Instruments 
VPEN system 

The VPEN is a multimedia Web annotation system developed by Hwang and Wang (2004), which is composed of 
features to annotate online learning materials, such as highlighting and underlining. It also affords the user the ability to 
write comments in text or in a sound-recording format as well as to read or share annotations. Figure 3 is an illustration 
of the use of the VPEN system. Meanwhile, the contents of annotations are recorded in a Microsoft® SQL server. 
Furthermore, each annotation contains useful information, such as the type and time of the annotation, which was used 
to identify if the annotation was made before class. 

Learning materials, pre-test, post-test, and questionnaire 

Learning materials in all classes include four lessons in a sixth-grade social course textbook. The pre-test and post-test 
consist of multiple-choice and open-ended questions, which are designed by the researcher and revised by domain 
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experts in order to exclude and modify ambiguous and unsuitable questions. The questionnaire, adapted from Selim 
(2003), was composed of open-ended questions and response questions with 5 dimensions, 32 items in a 5-point Likert 
scale, as shown in Appendix 1. 

RESULTS 

Reliability of the questionnaire 

SPSS software is used to analyze the reliability of the questionnaire. Cronbach’s alpha is employed to evaluate the 
internal consistency of each dimension of the questionnaire (1951). The result, Table 2, shows that all dimensions in the 
questionnaire are higher than 0.70. According to Wortzel (1979), the questionnaire would be highly reliable if the 
Cronbach alpha value is higher than 0.7; while it would be unreliable if it is less than 0.35. In other words, the reliability 
of the questionnaire is sufficiently high. 


Table 2.Questionnaire dimension and Cronbach alpha values 


# 

Dimension 

Cronbach Alpha Value 

1 

Perceived usefulness of VPEN 

.878 

2 

Perceived usefulness of pre-reading 

.897 

3 

Perceived usefulness of sharing annotation mechanism on learning 

.877 

4 

Perceived usefulness of sharing annotation mechanism on pre-reading 

.906 

5 

Perceived ease use of VPEN 

.766 


Validity of the questionnaire 

During the process of designing the questionnaire, the sentences of the items were modified for children of the same age 
and similar background to ensure understanding. The items were also verified and validated by experts. The ambiguous 
and unsuitable questions were removed, modified, and arranged in a proper procedure. 

Results of questionnaire analysis 

The goal of the questionnaire was to survey participants’ perceptions in five dimensions (i.e., usefulness of VPEN, 
usefulness of pre-reading, usefulness of the sharing annotation mechanism for learning, usefulness of the sharing 
annotation mechanism for pre-reading, and VPEN ease of use). The questionnaire was given to 93 participants (30 in 
the “Experiment”, 32 in “Control A,” and 31 in “Control B”), and 90 completed questionnaires were received (a 96 
percent return rate). The following are the perceived usefulness of three factors (pre-reading, VPEN and sharing 
mechanism) on learning and the perceived ease use of VPEN. 

Perceived usefulness of VPEN on enriching annotations 

As shown in Appendix 1, most participants strongly believed that VPEN can help them make annotations more fruitful 
by 67 percent. Furthermore, it was believed that VPEN can make it more convenient to annotate main points (e.g., 
marking, erasing, and updating the annotations) by 73.3 percent, and the tool makes referencing easier between 
annotations and online supplementary information by 81.7 percent. Similar opinions are also found in the open-ended 
question, “What do you think is the usefulness of VPEN on learning?” For example: 

“/ think that VPEN is helpful because there are lots of detailed online resources and it is more fruitful than a 
textbook. I could use online resources in VPEN when I made annotations. ” 

“It was always helpful. I use a Web-based dictionary to understand special terms and then write it down in 
VPEN. ” 

Meanwhile, Figure 3, sampled from VPEN tool logs, also demonstrates this statement. The fruit image in the textbook 
was not the fruit produced in the participant’s hometown at that time. Thus, the participant used the image of a local 
fruit (strawberry) to enrich the contents in Web-based learning material and made a text annotation that buying the cake 
decorated with the local season fruit (strawberries) would be cheaper, which matches with the learning goal: how to 
consume correctly. 
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Figure 3. A sample of enriching annotation 


However, there are a few opposing opinions that claim the Web-based annotation system (VPEN) is not convenient for 
learning, which was found in the open-ended question, “ What do you think is the usefulness of VPEN on learning?" For 
example: 

“My parents did not allow me to use the computer before the exam so I could not see annotations in VPEN. ” “No, 
I seldom use it because I couldn’t see annotations at home every day. ” 


Perceived usefulness of pre-reading on learning performance 

As shown in Appendix 1, overall the participants have positive responses to the usefulness of pre-reading on learning 
performance. Pre-reading is useful for them particularly in figuring out the main ideas in learning materials by 75.9 
percent, and in finding the ideas before class, which were not previously understood by 65.5 percent. Thus, 65.5 percent 
think pre-reading helps them to more efficiently understand the lecture in class and makes them more confident to 
perform well in class. It was believed that pre-reading is useful to learning and helpful to complete homework by 79.3 
percent. 

The perceived usefulness of pre-reading on listening efficiently during a lecture is also demonstrated by the opinions 
found in the open-ended question, “ What do you think is the usefulness of pre-reading on learning? For example: 

“It helped me memorize main ideas if I could study it before class. If I could find main ideas before class, then 1 
could more understand what my teacher taught in class. ” 

“In order to get better performance in pre-reading, I need to find more relative resources and it helped me absorb 
more knowledge and it was easier to make sense of what my teacher taught. Meanwhile I could quickly answer 
the questions that the teacher asked; it was so funny. ” 


However, a few opposing opinions were given against the usefulness of pre-reading. The following statements were 
written in the open-ended question, “What do you think is the usefulness of pre-reading on learning?" For example: 

“I think that one who could perform well is someone who has already been working well, and one who cannot 
perform well is someone who cannot work well initially. So pre-reading is not a key point. ” 

“What my teacher spoke in class is clearer than what was written in the textbook. So I think that I do not need to 
read before class. ” 


Although participants had positive responses to the usefulness of pre-reading on learning performance, some of them 
perceived that they needed more learning companions during pre-reading. Some examples were illustrated in the 
following statements, which were found in the open-ended question: “What do you need while you pre-read before 
class?” 


“I need some friends to discuss the learning materials while 1 pre-reading at home. ” 

“I need my parents to be with me while I pre-read at home...but my parents have no time to do it. Sometimes I 
want some friends to be with me while 1 pre-read. ” 

“I have tried my best to find some online resources related to the learning topic...I still want to see all of the 
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related resources found by all of my classmates but I did not want to ask them face-to-face because it is time- 
consuming... every time they update them or get new resources, I have to ask them again...I need my classmates to 
tell me their new or updated resources... but it is impossible. ” 

“...Pre-reading is really an effort-demanding job and that it is not easy to do by oneself”, which came from a 
participant in pre-reading with the paper-based annotation group. 

Perceived usefulness of sharing annotations mechanism on facilitating pre-reading and learning 
As shown in Appendix 1, overall the participants have a positive perception of the usefulness in sharing the annotation 
mechanism on facilitating pre-reading and learning. That is, sharing the annotation mechanism is useful for most of 
them; 71.5 percent of participants perceived that the sharing mechanism stimulates them to read classmates’ 
annotations, and 71.4 percent of the participants considered that reading classmates’ annotations helped them 
understand classmates’ learning process. A percentage of 60.7 felt that sharing stimulates them to make more 
annotations before class and 78.5 percent felt that sharing increased the quality of annotations made before the class. 
Thus, 78.6 percent of the participants think the sharing mechanism positively facilitates pre-reading, is helpful for their 
learning at a social course by 75 percent, and makes their learning easier by 71.7 percent. 

The perceived usefulness of the sharing annotation mechanism on pre-reading and learning is also revealed by the 
opinions found in the open-ended question, “What do you think is the usefulness of the sharing annotation mechanism 
on pre-reading and learning?” For example: 

“7 could share something I feel important for my friends via VPEN... remind them take care of the main idea in the 
learning materials. ...Sometimes I am happy to get an annotation, which has a written idea related the materials 
for me. ” 

“I mark some annotations for my friends when I found something wrong in their annotations. ” 

“I write some annotations by myself. Through the sharing mechanism I can view my friends annotations... I find 
some main ideas in it so I rewrite my annotations at home before class. ” 

“I know I could find some friends online...I am not alone...if I find someone’s annotation is valuable, I will 
repeatedly review his annotations and update my annotations in the other way. ” 

“The contents in classmates’ annotations sometimes contain a list of reference Web site addresses...the online 
resources will be a valued reference material for me. ’’ 

“7 could find useful information in my classmates’ annotations through the sharing annotations mechanism. 
Sometimes I could understand some concepts, which I did not realize initially through viewing classmates' 
annotations before class. Hence, it encouraged me to be more engaged in pre-reading.” 

In short, participants perceived that the sharing mechanism was useful for them in pre-reading and learning, presumably 
because of the cycle: annotations alone, viewing classmates’ annotations, and repairing original annotations. First, the 
participants annotate information by themselves. Second, they view the detailed information written in classmates’ 
annotations. Third, they repair and update the annotations written in the first step. The cycle will repeat again or it will 
go back to the second step. The participants pay attention to some classmates’ learning processes through repeatedly 
reviewing someone’s annotations and stimulating the annotations they made any time anywhere with computers, 
particularly the annotations made during pre-reading. Furthermore, the statements in the open-ended questionnaires 
seem to also reveal that most of the participants perceived that the VPEN sharing mechanism is not only an effective 
facilitator for pre-reading, but it also helps to form a pre-reading community for students, which encourages them to 
pre-read before class. 

Perceived ease of use of VPEN 

According to Appendix 1, 70 percent of participants think that the VPEN tool is easy to use. They particularly believe 
that they could easily find useful information in classmates’ annotations by 63.3 percent. However, around 36 percent of 
the participants thought that the process of learning how to operate the VPEN system was time consuming and VPEN 
was frustrating. The frustration stemmed from the Web environment and the VPEN Session protection mechanism; 
VPEN could not store annotations successfully due to users taking too long to act, according to interviews from 
participants who responded that they were frustrated. For example: 

“7 wrote some annotations in VPEN. Then I searched related information on the Web and it took me lots of time. 
When I came back, I continued to write some information in VPEN then I saved it. Some messages popped up. It 
told me, “Sorry! You are not logged in. ” I lost the annotations I wrote about 10 minutes earlier. ” 

In the future, a notification mechanism will be adopted to avoid the situation mentioned above. For example, the 
message, “No actions in VPEN long time would make you logout of VPEN’’. If it is written in the learning materials or 
warning system, it would be triggered before the activation of the VPEN Session protection mechanism, and it should 
be included in the VPEN system to warn users to take action. 
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Effects of pre-reading and the annotation tool on learning achievement 
Effects of individual pre-reading and the annotation tool in phase 1 

Based on the analytical results of two-way ANCOVA, as shown in Table 3, the findings indicate the following. 

First, the effect of a participant’s prior knowledge (pre-test) on learning achievement (post-test) was significant (F 
(mo) =20.357, pc.Ol). This means that the participants’ prior knowledge significantly affects participants’ learning 
achievement. Hence, employing ANCOVA to exclude the significant difference is the correct method. 

Second, the interaction between pre-reading and annotation tools is insignificant {F (I j 20 ) =.321, p>.05). It means 
that the effects of learning achievement caused by one factor were independent to another factor; therefore, we could 
directly explore the main effect of two factors. 

Third, although the mean of the post-test scores in the VPEN group (“Experiment” and “Control A,” M=78.61, 
N= 62) was higher than that in a paper-based annotation group(“Control B” and “Control C,” M=74.87, N= 63), there 
was no main effect for the annotation tool (F ( i, 12 o) =2.520, p>.05) while having controlled pre-test scores. It shows that 
the factor of the VPEN tool did not influence the students’ learning achievement. 

Fourth, the main effect on pre-reading is significant (F (U20 ) =12.921, pc.Ol), while controlled for the pre-test 
scores. The mean of the post-test scores in the pre-reading group (“Experiment” and “Control B,” M=80.11, /V=61) is 
higher than that in the group who did not request pre-reading (“Control A” and “Control C,” M= 73.50, V=64). It shows 
that learning achievement is significantly affected by pre-reading. The pre-reading factor influenced the learning 
achievement of a social course. 


Table 3. Results of two-way ANCOVA, pre-reading, and an annotation tool without a sharing mechanism in phase 1 


Source 

SS 

DF 

MS 

F 

P 

Covariance (score of pre-test) 

2087.344 

1 

2087.344 

20.357 ** 

.000 

Annotation tool 

258.377 

1 

258.377 

2.520 

.115 

Pre-reading 

1324.832 

1 

1324.832 

12.921 ** 

.000 

Interaction 

32.914 

1 

32.914 

.321 

.572 

Error 

12304.342 

120 

102.536 



Total 

16292.752 

124 





Note. **p<.01, *p <.05 

Effects of pre-reading and the sharing mechanism on learning achievement in phase 2 

Based on analytical results of the two-way ANCOVA, as shown in Table 4, the findings indicate the following. 

First, the effect of participants’ prior knowledge (pre-test) on learning achievement (post-test) is not significant (F 
(i,i 20 ) =1.954, p>. 05). It means that the participants’ prior knowledge did not significantly impact participants’ learning 
achievement. 

Second, the interaction between two factors was insignificant {F (II20) =.102, p>. 05). It revealed that the effect of 
one factor on a dependent variable (scores of post-test) would not be affected by another factor. Therefore, the main 
effect of two factors could be directly explored. 

Third, by employing a sharing annotation mechanism, the result indicates that a main effect of the annotation tool is 
F(i,i 20 ) =40.467, pc.Ol, which means that there is a statistical difference between the VPEN group (“Experiment” and 
“Control A”) and the paper-based annotation group (“Control B” and “Control C”). The mean score of the post-test 
(M=83.15, N= 62) in the VPEN group is higher than that in the paper-based annotation group (M=73.19, N= 63), which 
reveals that after employing a sharing mechanism, the learning achievements are significantly affected by an annotation 
tool with a sharing mechanism. 

Fourth, the results indicate that a main effect of pre-reading is F ( lil20 j =11.699, pc.Ol, which means that the pre- 
reading group and the group that did not ask for pre-reading have a statistical difference. The mean score of the pre- 
reading group was 80.8, while the group that did not pre-read was 75.58. In other words, learning achievements are 
significantly affected by pre-reading. The students would benefit from pre-reading and it facilitated the students’ 
learning achievement of a social course. 

In short, according to Table 3, it found that pre-reading significantly affect learning achievement, Web-based annotation 
tool or paper and pen annotation tool would not significantly affect learning achievement. However, according to Table 
4, after employing web-based sharing mechanism, Web-based annotation tool with sharing mechanism would 
significantly affect learning achievement. Meanwhile, the learning achievement (M= 86.2, V=30) of the pre-reading 
class with Web-based sharing annotation (“Experiment”) was significantly higher than the learning achievement 
(M=75.58, V=31) of traditional pre-reading class with paper and pen annotation tool (“Control B”) (r=4.862, pc.Ol). 
Thus, implementing a pre-reading sharing mechanism with Web-based annotation enlarge the effectiveness of pre- 
reading on learning achievement. 
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Table 4. Results of two-way ANCOVA, pre-reading, and an annotation tool with a sharing mechanism in phase 2 


Source 

SS 

DF 

MS 

F 

P 

Covariance(score of pretest) 

155.762 

1 

155.762 

1.954 

.165 

Annotation tool 

3226.111 

1 

3226.111 

40.467 ** 

.000 

Pre-reading 

932.624 

1 

932.624 

11.699 ** 

.001 

Interaction 

8.115 

1 

8.115 

.102 

.750 

Error 

9566.555 

120 

79.721 



Total 

13709.952 

124 





Note. **p<.01, *p<.05 

Correlation between learning achievement and the quantity of annotations in pre-reading 

According to Table 3 and Table 4, the learning achievement in the pre-reading group scored significantly higher than the 
group that did not employ pre-reading. Thus, it will be necessary to further investigate the correlation between learning 
achievements and the quantity of pre-reading annotations in different types of annotations (i.e., explicit and inexplicit 
annotations) in the pre-reading group while using the Pearson correlation and an interview. 

Correlation between learning achievement and the quantity of inexplicit annotation in pre-reading 
The results of two phases indicate that there were no positive relationships between learning achievement and the 
quantity of inexplicit annotation in pre-reading (phasel: r=.033, p>. 05; phase2: r=.086, p>. 05). With no textual 
comment cues, it is purely a heap of symbols consisting of lines and highlights and students may forget the meaning of 
the inexplicit annotation, according to the following statement made by a student who made inexplicit annotations. 

“Sometimes I forget the meanings of the underlines. It might be a main point or something I have not reviewed.'’’’ 

Correlation between learning achievement and the quantity of explicit annotation in pre-reading 
All phases exist as a positive correlation between learning achievement and explicit annotation in pre-reading (phasel: 
r=.582, pc.Ol; phase2: r=.615, p<.01). Hence, those who made more explicit annotations during pre-reading also 
experienced greater learning achievement. It revealed that students could attain greater learning achievement if they 
make enough preparation via explicit annotations before class. 

Meanwhile, an interesting phenomenon was found during the interview. Although annotations in class are important 
(Lin, 2006), making too many annotations in class would take much time to encode the lecture into annotations; 
therefore, it may interfere with students’ ability to carefully listen to the lecture. An interviewee made the following 
supporting statement: 

“The speed of the lecture is far beyond the speed of my writing. For me, I need more time to think in class before 
taking notes. Taking too much time to annotate means that I would take too much time thinking and it would 
prevent me from listening to the lecture carefully. ” 

Furthermore, during the lecture, students would carefully listen to the main ideas that they did not understand before 
class. A student who was engaged in a pre-reading exercise mentioned the following supporting statement: 

“7 read the materials before class and I highlight something important and write down my answers following my 
teacher’s question list. During the lecture, what I need to do is listening to the part I did not understand initially. 
Then before the exam, I review the main points and the feelings I wrote. I found it was so clear and so many key 
points were made that it was not needed to find others. ” 

A statement from someone who did not pre-read reinforces this phenomenon: 

“/ did not know what message was important because my teacher made lots of key points. Do I need to write it 
down or memorize it? I know lots of important information given by my teacher was lost during the lecture. In 
order to prepare for the exam, I need more time to read the materials after class, but some of them I still could not 
figure out. ” 

It seems that pre-reading would affect the student’s learning behavior in class or after class. Thus, the effect of pre- 
reading on the different processes (in class and after class) should be investigated in a further study. 

The effects of pre-reading modes with Web-based sharing mechanism on the quantity of annotation 

It is evident that a student’s learning achievement is significantly affected by pre-reading (as shown in Table 3 and 
Table 4) and the Web-based sharing mechanism (as shown in Table 4). It is necessary to further investigate the effects 
of pre-reading modes with Web-based sharing mechanism on the quantity of annotation. Due to the difficulty of 
calculating the quantity of annotations in the group that used paper and pen, “Control B’’ and “Control C,” this study 
only analyzes the quantity of annotations in the VPEN group (the “Experiment” and “Control A”), a total of 62 
participants. 

According to Table 5, a significant difference exists between the classes with/without pre-reading in two phases. It 
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seemed that those who engaged in pre-reading made more annotations than those who did not engage in pre-reading. 
Thus, pre-reading plays an important role in facilitating students to be more engaged in reading materials and making 
annotations. 

By employing a sharing mechanism, as seen in Table 5, the quantity of annotations increased in the pre-reading class, 
while the quantity of annotations decreased in another class without a request for pre-reading. Thus, the gap of the 
quantity of annotations between the two classes became larger due to the sharing mechanism in phase 2. According to 
the findings in the questionnaires and open questions, it is revealed that the sharing mechanism can help to form a pre- 
reading community for students, which encourages them to make more annotations before class. Thus, pre-reading with 
a sharing mechanism could be a promising and effective way to stimulate students to engage in more pre-reading 
exercises. 


Table 5. Total Annotations 



With Pre-reading 

Without Request for Pre- 
reading 

t-value 


M 

SD 

M 

SD 


Phasel 

38.47 

19.158 

27.13 

16.597 

2.496** 

Phase2 

46.07 

22.228 

25.28 

14.645 

4.374** 

N 

30 


32 




Note. **p<.01 

M = mean; SD= standard deviation 

Usefulness of the instructor’s perception toward pre-reading modes with VPEN 

The interview was applied in order to explore the instructor’s attitude toward pre-reading. In order to stimulate the 
instructor’s recall, the online annotations made by participants and the instructor were used during the interview. 
Moreover, the instructor thought the students’ annotations made in pre-reading helped him to understand the students’ 
prior knowledge and preparation before class, particularly from students’ annotations labeled “uncomprehending’ and 
the annotations in response to the pre-reading guided questions. Furthermore, even the annotations, which students 
labeled “understanding,” revealed their preconceptions toward the main ideas on learning materials. The following are 
the interview contents of the instructor, in the interview outline, “ What do you think is the usefulness of pre-reading 
modes with VPEN on your instruction? 

“In my opinions, I found that it was a rapid way to realize students’ preparation and prior knowledge via the 
follow steps. First, I get an outline about who studied hard or who did not prepare before class via the quantity of 
annotations made by students before class, which is located at the bottom in VPEN. Second, I rapidly get a 
preliminary understanding on the contents, which most students feel are difficult, through annotations labeled 
‘uncomprehending’ by students on Web-based learning material. Third, through the content of annotations and 
the answers for questions, which were used to guide students' reading before class, help me get a more detailed 
understanding of their preconception about investment activities. Overall, through the three steps I found that 
most students understand that investment activities are activities for a profit fund and most of them illustrated an 
example of stock....Although students labeled that he/she knows the meanings of the paragraph, his/her 
annotations revealed that he/she has a poor understanding toward the investment activities.... For example, some 
students confused the concept between investment activities and the job they want to do in the future or what the 

meaning is of the decline of the stock . Or, they think that investment activities refer to getting a profit before 

investment. Meanwhile, except for the stock investment, most students could not show me other investment 
examples and some of them know something about being careful before making investments but they have no idea 
about how to do it or where they could get the information to help them learn about being careful....Iprepared a 
sheet to describe my investment activities before class and prepared an activity for them to investigate their 
parents ’ jobs and investment activities to help them understand what investment activities are and how to relate 
them with their lives. ” 

In short, the instructor perceived usefulness toward pre-reading modes with the VPEN tool, presumably because he 
could understand students’ prior knowledge through the following tricks: the screen presenting each student’s quantity 
of annotations (and illustrated as Figure 4), the contents labeled “uncomprehending” marked by the students, the 
students’ answers written in the pre-reading guided questions, and the detailed content written in annotations made by 
students. Through the tricks, the instructor knows there is a need to arrange an activity about investigating the parents’ 
jobs and their investment activities to help students understand how many investment activities relate to their lives, not 
only the stocks and the difference between jobs and investment activities. Thus, it helps the instructor to prepare well in 
advance for the lecture. 


Copyright © The Turkish Online Journal of Educational Technology 


244 










TOJET: The Turkish Online Journal of Educational Technology - April 2011, volume 10 Issue 2 


Switch User jg: 

30 

□ Group mode 

[ Member 
60- (76, 

6C (23. 
6C (18 

60 ■ (65 
6C (67 

ListfWMii] 

0,11) 

0.0) 

0.3) 

0.0) 

p.O) 

+ 1 

4 

Format: us 
(text,imag 

textMarks 
+ underlig 

erName 

i,cmt) 

= highlights 
hts marks 




This is a sample of annotations made by a student before class in the guiding 
questions. 

Page 60-61 : 


Author: 60** 

(67,0,0) 

(Read-only) 


Q VPen System 

O HL UL Erase yml 


mm- »| 


fr WlDTOSS ? ? 

■ shkwk ■ mtx&fsmmn ■ ■ MUmmmma, • (siiwsesMss- 

ramutMSMMa • ffiTOisitttajswo - 


The screen presents each student’s quantity of annotations, which is located at the bottom in VPEN. 

The users click the “Switch User” list to view each classmate’s annotations, as showed in the above. 

Figure 4. A sample of the sharing mechanism for viewing classmates’ annotations 

CONCLUSIONS AND DISCUSSION 

This study is aimed at exploring students’ annotation behaviors and uncovers the relationship between these factors, 
such as pre-reading, annotation tools, and sharing scenarios, quantity of annotations, and learning achievement. Due to 
limited reference studies found in this area and to avoid careless use of technology in education, a four-class and two- 
phase research was conducted in order to identify the interplay of pre-reading with a Web-based annotation tool. The 
analysis of learning performances and questionnaires revealed the following about the effectiveness of the VPEN with a 
sharing annotation mechanism in pre-reading activities. 

First, learning achievement was significantly affected by pre-reading and pre-reading with a Web-based sharing 
mechanism was an effective way to help students be ready to learn. As the results from this study showed, the explicit 
annotations in the pre-reading exercises with a Web-based annotation tool had a significantly positive correlation with 
learning achievement. Meanwhile, the learning achievement in the pre-reading group scored significantly higher than 
the group that did not employ pre-reading and the learning achievement of the pre-reading class with Web-based sharing 
annotation was significantly higher than the learning achievement of traditional pre-reading class with paper and pen 
annotation approach. Furthermore, according to the questionnaire, the participants have an overall positive perception of 
the usefulness of a sharing annotation mechanism on enhancing pre-reading and learning. Participants perceived that an 
integrated VPEN tool with a sharing mechanism for pre-reading was an effective way to stimulate them to figure out 
more main points and to perceive more incomprehensible main points, which may make them pay more attention during 
a lecture. Thus, pre-reading with VPEN could be a useful learning strategy. We strongly emphasize that an integrated 
technology (a Web-based annotation tool with a sharing mechanism) in pre-reading was an effective way to help 
students prepare to learn. 

Second, participant pre-reading with a Web-based annotation tool could reveal preparation before class and prior 
knowledge, thereby helping the instructor to prepare well in advance for the lecture. Through annotations made by 
participants in pre-reading via a Web-based annotation tool, the instructor could understand participants’ preparation 
and prior knowledge without offering a pre-test, which could help the instructor prepare well for the lecture, as 
mentioned in the interview. The annotations also implied that in the future there is a need to further examine whether 
adaptive learning could be applied according to such prior knowledge, which comes from the annotations written by 
participants. 

Third, participants perceived the sharing annotation mechanism as an effective facilitator for pre-reading, stimulating 
more Web-based annotations before class, and thereby producing significantly better learning effects. The results from 
the questionnaire revealed that most of the participants in pre-reading classes were encouraged to continue pre-reading 
through the VPEN’s sharing annotation mechanism. Meanwhile, the result in this study also found that the gap of the 
quantity of annotation between the classes with/without pre-reading from phase 1 to phase 2 was expanded by the 
sharing mechanism. Furthermore, the Web-based annotation tool with a sharing annotation significantly affected 
learning achievement. Thus, it implied that a sharing mechanism could facilitate pre-reading and stimulate more Web- 
based annotation, thereby producing a significantly better effect on learning. 

Integrating a Web-based sharing mechanism into pre-reading is an essential way to facilitate pre-reading to stimulate 
more annotations before class and to prompt participants to overcome the demanding work of pre-reading through the 
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pre-reading community. Additionally, an integrated Web-based sharing mechanism produces significantly better effects 
on learning. In this research, the effectiveness of pre-reading, the annotation tool, and the sharing mechanism on 
learning was explored. However, how to conduct efficient pre-reading was not evaluated in this study. In the future, 
more advanced mechanisms, such as self-regulation learning, will be studied to encourage students to more efficiently 
manage their own learning before class. 
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Appendix 1 Questionnaire Results 

Dimension 1: Perceived usefulness of the annotation tool: (“Experiment” and “Control A” respond, a total of 62 
participants with 60 responses) 




SA 

A 

UD 

DA 

SD 

M 

1. 

I feel the Web-based annotation tool is helpful for learning a 
social course. 

43.3 

31.7 

20 

5 

0 

4.13 

2. 

I feel the Web-based annotation tool is useful for completing 
homework at a social course. 

41 

35 

21.7 

1.7 

0 

4.17 

3. 

I feel the Web-based annotation tool makes the learning easier 
at a social course. 

35 

35 

26.7 

3.3 

0 

4.02 

4. 

I feel the Web-based annotation tool makes annotating main 
points more convenient at a social course. 

38.3 

35 

21.7 

5 

0 

4.07 

5. 

I feel the Web-based annotation tool enriches the contents of 
annotations. 

41 

26 

30 

1.7 

0 

4.08 

6. 

I feel the Web-based annotation tool makes the reference more 
convenient between the contents of annotations and online 
supplementary information. 

40 

41.7 

18.3 

0 

0 

4.22 

Dimension 2: Perceived usefulness of pre-reading: (“Experiment” 

and “Control B” respond, a total of 

61 participants with 58 responses) 









SA 

A 

UD 

DA 

SD 

M 

1. 

I feel pre-reading is helpful for learning a social course. 

31 

48.3 

17.2 

3.4 

0 

4.07 

2. 

I feel pre-reading is useful for completing homework at a social 
course. 

27.6 

51.7 

19 

1.7 

0 

4.05 

3. 

I feel pre-reading makes the learning easier at a social course. 

32.8 

41.4 

20.7 

3.4 

1.7 

4 

4. 

I feel pre-reading lets me have more confidence in performing 
well at a social course. 

31 

34.5 

24.1 

10.3 

0 

3.86 

5. 

I feel pre-reading makes me more efficient in understanding the 
lecture. 

22.4 

43.1 

29.3 

5.2 

0 

3.83 

6. 

I think that pre-reading is helpful for figuring out the main 
points of the learning materials. 

27.6 

48.3 

19 

3.4 

1.7 

3.97 

7. 

I think that pre-reading is helpful for finding the 
incomprehensible contents before class. 

22.4 

43.1 

29.3 

5.2 

0 

3.83 


Dimension 3: 

Perceived usefulness of the sharing annotations mechanism on learning: (only “Experiment” and 
“Control A” respond, a total of 62 participants with 60 responses) 




SA 

A 

UD 

DA 

SD 

M 

1. 

I think Web-based sharing annotation mechanism is really 
helpful for learning a social course. 

35 

40 

23.3 

1.7 

0 

4.08 

2. 

I think Web-based sharing annotation mechanism is useful for 
completing homework at a social course. 

31.7 

41.7 

20 

5 

1.7 

3.97 

3. 

I think Web-based sharing annotation mechanism makes the 
learning easier at a social course. 

36.7 

35 

25 

3.3 

0 

4.05 

4. 

I think Web-based sharing annotation mechanism stimulates me 
more engaged in making annotations. 

33.3 

25 

40 

1.7 

0 

3.9 

5. 

I think the Web-based sharing annotation mechanism 
encourages me to read classmates’ annotations. 

33.3 

38.3 

21.7 

6.7 

0 

3.98 

6. 

I think the Web-based sharing annotation mechanism is useful 

31.7 

41.7 

21.7 


0 



for understanding the annotations made by classmates. 

D 


7. 

I think the Web-based sharing annotation mechanism could 
improve the quality of the contents of my annotations. 

35 

35 

28.3 

1.7 

0 

4.03 

Dimension 4: 







Perceived usefulness of the sharing annotation mechanism on pre-reading: (only 
responds, a total of 30 participants with 28 responses) 

“Experiment’ 





SA 

A 

UD 

DA 

SD 

M 

1. 

I think that the sharing mechanism is helpful for pre-reading. 

35.7 

42.9 

21.4 

0 

0 

4.14 

2. 

I think that reading classmates’ annotations encourages me to 
be more engaged in pre-reading. 

25 

42.9 

28.6 

3.6 

0 

3.89 
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3. 

I think that the sharing mechanism is helpful for rapidly 
completing the pre-reading ; 

28.6 

35.7 

32.1 

3.6 

0 

3.89 

4. 

I think that that the sharing mechanism stimulates me to be 
more engaged in making more annotations. 

32.1 

28.6 

39.3 

0 

0 

3.93 

5. 

I think that the sharing mechanism stimulates me to read the 
annotations made by classmates during pre-reading. 

28.6 

42.9 

25 

3.6 

0 

3.96 

6. 

I think that reading classmates’ annotations could stimulate me 
to make more annotations during pre-reading. 

17.9 

50 

25 

7.1 

0 

3.79 

7. 

I think that reading classmates’ annotations could improve the 
quality of annotations made before class. 

32.1 

46.4 

14.3 

7.1 

0 

4.04 

8. 

I think that the sharing mechanism was helpful for 
understanding classmates’ learning process through their 
annotations made during pre-reading. 

25 

46.4 

17.9 

10.7 

0 

3.86 

Dimension 5: 







The ease of use of the web-based annotation tool (only “Experiment’ 

’ and 

“Control A” 

respond, a total 

of 62 participants with 60 responses) 









SA 

A 

UD 

DA 

SD 

M 

1 . 

I think that using the Web-based annotation tool is very easy for 
me. 

33.3 

36.7 

20 

6.7 

3.3 

3.9 

2. 

I think that the process of learning how to use the Web-based 
annotations tool is time-consuming. 

23.3 

13.3 

35 

25 

3.3 

3.28 

3. 

I think that finding useful information from the annotations 
made by classmates is very easy for me. 

30 

33.3 

31.7 

5 

0 

3.88 

4. 

I think that the using of the Web-based annotation tool 
frustrates me. 

21.7 

15 

33.3 

26.7 

3.3 

3.25 


Note. SA = strong agree; A = agree; UD = undecided; DA = disagree; SD = strong disagree; M = mean 

The number in the table means the percentage of each column. For example, the number of the fourth item in dimension 

5 at SA column means that 21.7 percent of the participants strongly agree. 

Open-ended questions: 

1. What do you think is the usefulness ofVPEN on learning? 

2. What do you think is the ease use ofVPEN? 

3. What do you think is the usefulness of pre-reading on learning? 

4. What do you need while you pre-read before class? 

5. What do you think is the usefulness of sharing annotation mechanism on pre-reading and learning? 
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